{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# PyTorch 基础 : 自动求导\n",
    "深度学习的算法本质上是通过反向传播求导数，而PyTorch的autograd模块则实现了此功能。在Tensor上的所有操作，autograd都能为它们自动提供微分，避免了手动计算导数的复杂过程。\n",
    "\n",
    "***从0.4起, Variable 正式合并入Tensor, Variable 本来实现的自动微分功能，Tensor就能支持。读者还是可以使用Variable(tensor), 但是这个操作其实什么都没做。***\n",
    "\n",
    "所以，以后的代码建议直接使用Tensor，因为官方文档中已经将Variable设置成过期模块\n",
    "\n",
    "要想使得Tensor使用autograd功能，只需要设置tensor.requries_grad=True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4.1'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 首先要引入相关的包\n",
    "import torch\n",
    "#打印一下版本\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在张量创建时，通过设置 requires_grad 标识为Ture来告诉Pytorch需要对该张量进行自动求导，PyTorch会记录该张量的每一步操作历史并自动计算"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.8647, 0.9577, 0.7270, 0.4148, 0.8135],\n",
       "        [0.5607, 0.3536, 0.4799, 0.5256, 0.6381],\n",
       "        [0.3390, 0.7099, 0.8135, 0.0005, 0.0769],\n",
       "        [0.9587, 0.5234, 0.3292, 0.9823, 0.3233],\n",
       "        [0.3847, 0.6815, 0.5941, 0.7510, 0.0381]], requires_grad=True)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(5, 5, requires_grad=True)\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.0591, 0.7597, 0.7404, 0.6204, 0.4741],\n",
       "        [0.7539, 0.1664, 0.9127, 0.8678, 0.6723],\n",
       "        [0.5833, 0.5544, 0.6002, 0.2238, 0.6560],\n",
       "        [0.6884, 0.6652, 0.0063, 0.4368, 0.4200],\n",
       "        [0.2989, 0.1864, 0.1204, 0.8179, 0.8159]], requires_grad=True)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = torch.rand(5, 5, requires_grad=True)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们看到 该张量的grad_fn已经被赋予了一个新的函数。下面我们来调用反向传播函数，计算其梯度"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(26.9424, grad_fn=<SumBackward0>)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z=torch.sum(x+y)\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 简单的自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "z.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n",
      "tensor([[1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.],\n",
      "        [1., 1., 1., 1., 1.]])\n"
     ]
    }
   ],
   "source": [
    "#看一下x和y的梯度\n",
    "print(x.grad)\n",
    "print(y.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 复杂的自动求导"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor([[0.5413, 0.1917, 0.7621, 0.2361, 0.5535],\n",
       "        [0.7332, 0.2089, 0.3866, 0.6624, 0.0793],\n",
       "        [0.3837, 1.2926, 1.2701, 0.9063, 1.0719],\n",
       "        [0.7641, 0.0275, 0.9585, 0.0742, 0.2558],\n",
       "        [0.4969, 0.6638, 1.0299, 0.4037, 0.5146]], grad_fn=<ThAddBackward>)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = torch.rand(5, 5, requires_grad=True)\n",
    "y = torch.rand(5, 5, requires_grad=True)\n",
    "z= x**2+y**3\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[0.4281, 0.6938, 1.7237, 0.9432, 1.4095],\n",
      "        [0.5688, 0.4556, 1.2410, 1.4190, 0.4931],\n",
      "        [1.2061, 1.4807, 1.3298, 1.2998, 0.5579],\n",
      "        [0.3873, 0.2911, 1.7443, 0.5431, 1.0106],\n",
      "        [0.8796, 1.6243, 1.9457, 1.0224, 0.3311]])\n"
     ]
    }
   ],
   "source": [
    "#我们的返回值不是一个scalar，所以需要输入一个大小相同的张量作为参数，这里我们用ones_like函数根据x生成一个张量\n",
    "z.backward(torch.ones_like(x))\n",
    "print(x.grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([ 9., 18., 39.], grad_fn=<ThAddBackward>)\n",
      "tensor([ 3., 12., 27.])\n",
      "tensor([2., 2., 2.])\n"
     ]
    }
   ],
   "source": [
    "# 这个例子更好理解复杂的求导\n",
    "a = torch.tensor([1., 2., 3.],requires_grad=True)\n",
    "b = torch.tensor([4., 5., 6.],requires_grad=True)\n",
    "z = a**3 + 2*b\n",
    "print(z)\n",
    "z.backward(torch.ones_like(a))\n",
    "print(a.grad) # 即 3*a^2\n",
    "print(b.grad) # 即 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用`with torch.no_grad()`上下文管理器临时禁止对已设置requires_grad=True的张量进行自动求导。这个方法在测试集计算准确率的时候会经常用到，例如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "False\n"
     ]
    }
   ],
   "source": [
    "with torch.no_grad():\n",
    "    print((x +y*2).requires_grad)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
